client: support to get all keyspaces #6793

ystaticy · 2023-07-13T01:46:12Z

What problem does this PR solve?

Issue Number: close #6733

What is changed and how does it work?

Check List

Tests

Unit test

Code changes

Add a pdclient func,and RPC interface.

Release note

We only use get all keyspaces and don't need to watch keyspace changes.
Removed WatchKeyspaces in pd client to avoid excessive use of goroutine.

Signed-off-by: ystaticy <[email protected]>

ti-chi-bot · 2023-07-13T01:46:14Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

lhy1024
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

ti-chi-bot · 2023-07-13T01:46:22Z

Hi @ystaticy. Thanks for your PR.

I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: ystaticy <[email protected]>

client/keyspace_client.go

Signed-off-by: ystaticy <[email protected]>

ystaticy · 2023-07-13T03:26:14Z

/test build

ti-chi-bot · 2023-07-13T03:26:28Z

@ystaticy: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test build

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lhy1024 · 2023-07-13T03:28:37Z

client/keyspace_client.go

@@ -153,3 +155,35 @@ func (c *client) UpdateKeyspaceState(ctx context.Context, id uint32, state keysp

 	return resp.Keyspace, nil
 }
+
+// GetAllKeyspaces get all keyspaces metadata.
+func (c *client) GetAllKeyspaces(ctx context.Context, startID uint32, limit uint32) ([]*keyspacepb.KeyspaceMeta, error) {


If replace WatchKeyspaces with GetAllKeyspaces, will it cost too much? Do we need to use watch loop in server?

lhy1024 · 2023-07-13T03:29:30Z

/ok-to-test

ystaticy · 2023-07-13T03:32:21Z

/test build

ystaticy · 2023-07-13T03:37:46Z

/retest

Signed-off-by: ystaticy <[email protected]>

ystaticy · 2023-07-13T04:47:47Z

/test

ti-chi-bot · 2023-07-13T04:48:02Z

@ystaticy: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

/test build
/test integration-copr-test

Use /test all to run the following jobs that were automatically triggered:

tikv/pd/ghpr_build

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ystaticy · 2023-07-13T04:48:57Z

/test all

Signed-off-by: ystaticy <[email protected]>

codecov · 2023-07-13T05:29:07Z

Codecov Report

Merging #6793 (44d4466) into master (4a4c567) will decrease coverage by 0.09%.
The diff coverage is 54.54%.

❗ Current head 44d4466 differs from pull request most recent head 5b164c6. Consider uploading reports for the commit 5b164c6 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6793      +/-   ##
==========================================
- Coverage   74.25%   74.17%   -0.09%     
==========================================
  Files         413      413              
  Lines       43399    43410      +11     
==========================================
- Hits        32225    32198      -27     
- Misses       8293     8334      +41     
+ Partials     2881     2878       -3

Flag	Coverage Δ
unittests	`74.17% <54.54%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

client/keyspace_client.go

Signed-off-by: ystaticy <[email protected]>

client/go.mod

Signed-off-by: ystaticy <[email protected]>

lhy1024 · 2023-07-13T07:49:28Z

server/keyspace_service.go

+	}
+
+	manager := s.GetKeyspaceManager()
+	keyspaces, err := manager.LoadRangeKeyspace(request.StartId, int(request.Limit))


How often will this interface be called, and will it cause additional performance issues if all keyspace is loaded from etcd each time? How about using watch loop?

lhy1024 · 2023-07-13T07:52:13Z

/merge

ti-chi-bot · 2023-07-13T07:52:15Z

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-07-13T07:52:17Z

This pull request has been accepted and is ready to merge.

Commit hash: 0bd4030

ti-chi-bot · 2023-07-13T07:52:29Z

@ystaticy: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ystaticy added 5 commits July 12, 2023 22:25

get all keyspace

03ace6b

Signed-off-by: ystaticy <[email protected]>

get all keyspace

b1f49bd

Signed-off-by: ystaticy <[email protected]>

get all keyspace

d5e25a0

Signed-off-by: ystaticy <[email protected]>

get all keyspace

ef4d153

Signed-off-by: ystaticy <[email protected]>

get all keyspaces

ef21a7e

Signed-off-by: ystaticy <[email protected]>

ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. labels Jul 13, 2023

Merge branch 'master' into get_all_keyspaces

1efc536

ti-chi-bot bot added the needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. label Jul 13, 2023

ti-chi-bot bot requested review from HunDunDM and rleungx July 13, 2023 01:46

ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 13, 2023

ystaticy marked this pull request as ready for review July 13, 2023 01:50

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 13, 2023

get all keyspaces

5055bea

Signed-off-by: ystaticy <[email protected]>

rleungx approved these changes Jul 13, 2023

View reviewed changes

client/keyspace_client.go Outdated Show resolved Hide resolved

ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 13, 2023

ystaticy added 2 commits July 13, 2023 10:31

get all keyspaces

19a681f

Signed-off-by: ystaticy <[email protected]>

get all keyspaces

ec844f3

Signed-off-by: ystaticy <[email protected]>

ti-chi-bot bot removed the do-not-merge/needs-linked-issue label Jul 13, 2023

get all keyspaces

4c038da

Signed-off-by: ystaticy <[email protected]>

lhy1024 reviewed Jul 13, 2023

View reviewed changes

lhy1024 changed the title ~~Get all keyspaces~~ client: support to get all keyspaces Jul 13, 2023

ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Jul 13, 2023

get all keyspaces

68287bb

Signed-off-by: ystaticy <[email protected]>

get all keyspaces

69e48d4

Signed-off-by: ystaticy <[email protected]>

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 13, 2023

ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jul 13, 2023

lhy1024 reviewed Jul 13, 2023

View reviewed changes

client/keyspace_client.go Show resolved Hide resolved

get all keyspaces

44d4466

Signed-off-by: ystaticy <[email protected]>

lhy1024 approved these changes Jul 13, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jul 13, 2023

lhy1024 reviewed Jul 13, 2023

View reviewed changes

client/go.mod Outdated Show resolved Hide resolved

update kvproto

0bd4030

Signed-off-by: ystaticy <[email protected]>

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 13, 2023

lhy1024 reviewed Jul 13, 2023

View reviewed changes

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 13, 2023

Merge branch 'master' into get_all_keyspaces

5b164c6

ti-chi-bot bot merged commit 42ce6a1 into tikv:master Jul 13, 2023

ystaticy mentioned this pull request Dec 6, 2023

GC V2 interfaces #7500

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

client: support to get all keyspaces #6793

client: support to get all keyspaces #6793

ystaticy commented Jul 13, 2023 •

edited by lhy1024

Loading

ti-chi-bot bot commented Jul 13, 2023 •

edited

Loading

ti-chi-bot bot commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

lhy1024 Jul 13, 2023

lhy1024 commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ystaticy commented Jul 13, 2023

codecov bot commented Jul 13, 2023 •

edited

Loading

lhy1024 Jul 13, 2023

lhy1024 commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

client: support to get all keyspaces #6793

client: support to get all keyspaces #6793

Conversation

ystaticy commented Jul 13, 2023 • edited by lhy1024 Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Jul 13, 2023 • edited Loading

ti-chi-bot bot commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

lhy1024 Jul 13, 2023

Choose a reason for hiding this comment

lhy1024 commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ystaticy commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ystaticy commented Jul 13, 2023

codecov bot commented Jul 13, 2023 • edited Loading

Codecov Report

lhy1024 Jul 13, 2023

Choose a reason for hiding this comment

lhy1024 commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ti-chi-bot bot commented Jul 13, 2023

ystaticy commented Jul 13, 2023 •

edited by lhy1024

Loading

ti-chi-bot bot commented Jul 13, 2023 •

edited

Loading

codecov bot commented Jul 13, 2023 •

edited

Loading